Route optimization using measured congestion

ABSTRACT

A flow based routing method and apparatus selects a path from a plurality of different paths for assignment to a flow. The path is selected based on traffic performance measurements which identify relative congestion and performance of the different paths, so that traffic flows can be diverted away from network congestion points, thereby allowing network resources to be load balanced at a flow granularity. The present invention may be configured on physical or virtual links on an NE to enhance the forwarding of packets using the primary link and one or more alternate links to any given destination.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.11/530,579, filed on Sep. 11, 2006, which is a continuation-in-part ofpatent application Ser. No. 11/251,252, filed Oct. 14, 2005 (abandoned),application Ser. No. 11/530,579 claims the benefit of U.S. ProvisionalPatent Application No. 60/716,181, filed Sep. 12, 2005, and applicationSer. No. 11/251,252 claims priority to U.S. Provisional PatentApplication No. 60/708,963, filed Aug. 17, 2005. Each applicationidentified in this paragraph is incorporated herein by reference in itsentirety.

BACKGROUND

The present disclosure relates generally to the field of networking andmore particularly to a method and apparatus for directing traffic aroundcongestion in a network.

Networks that need to support real-time inelastic services such as voiceand video may need a controlled environment that allows guarantees onthe quality of service to be provided to traffic. One method of ensuringthat quality of service constraints can be met is to limit traffic onthe network to an amount which is well below the network bandwidthlimit. Such a solution is not cost-effective as it blocks informationfrom being transmitted on the network although other paths areunderutilized and does not allow the full capacity of the network to beutilized. One method that is used to ensure that a network is fullyutilized is to introduce traffic into the network onto different pathsuntil the network becomes congested and packets are lost. However, lostpackets adversely affect the quality of service realized by traffic inthe network. Because real-time traffic is particularly sensitive topacket loss and delay it is desirable to detect congestion beforepackets are dropped or significantly queued and quality of service isadversely affected.

BRIEF SUMMARY

According to one aspect of the present disclosure, a method is providedfor selecting a path for a flow comprising a plurality of packets. Anetwork element performs traffic performance measurements on a primarypath to determine congestion, Quality of Service (QoS), and connectivityand selectively routes the flow to an outbound link around the primarypath in response to a comparison of the congestion, QoS, andconnectivity of the primary path with congestion, QoS, and connectivityof an alternate path. As a result, traffic flows are diverted away fromcongested paths at a flow granularity. With such an arrangement, methodsin accordance with the present disclosure enable load sharing of flowsbetween multiple paths based on measured network congestion to ensurethat a desired quality of service may be received by the flow.

According to a further aspect of the present disclosure, a networkdevice comprises real-time traffic performance measurement logic forperforming traffic performance measurements on a primary path todetermine congestion, Quality of Service (QoS), and connectivity and forselectively routing a flow to an outbound link around the primary pathin response to a comparison of the congestion, QoS, and connectivity ofthe primary path with congestion, QoS, and connectivity of an alternatepath. With such an arrangement, quality of service guarantees can be metbecause flows are not mapped to paths that are congested.

These and other advantages of the present disclosure will be describedwith respect to the attached Figures.

BRIEF DESCRIPTION OF THE DRAWINGS

So the manner in which the above recited features of the presentdisclosure may be understood in detail, a more particular description ofembodiments of the present disclosure, briefly summarized above, may behad by reference to embodiments, which are illustrated in the appendeddrawings. It is to be noted, however, the appended drawings illustrateonly typical embodiments encompassed within the scope of the presentdisclosure, and, therefore, are not to be considered limiting, for thepresent disclosure may admit to other equally effective embodiments,wherein:

FIGS. 1A, 1B, and 1C are network diagrams that are used to describe amethod in accordance with the present disclosure for directing trafficflows around points of congestion in the network;

FIG. 2 is a block diagram of several representative components that maybe included in forwarding logic of a network element in accordance withthe present disclosure; and

FIG. 3 is a flow diagram illustrating exemplary steps that may beperformed by the forwarding logic of FIG. 2 to route traffic flows awayfrom congested points in the network.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of embodiments orother examples described herein. In some instances, well-known methods,procedures, components and circuits have not been described in detail,so as to not obscure the following description. Further, the examplesdisclosed are for illustrative purposes only and should not be construedas limiting of the scope of embodiments of the present disclosure.

Voice over Internet Protocol (VoIP) is the convergence of traditionalvoice delivery on the data, or packet switched, network. VoIP protocolshave two components: a control flow and a data flow. The control flowuses protocols such as Session Initiation Protocol (SIP), H.225 (forH.323), or Media Gateway Control Protocol (MGCP). The data flow or“media” component uses Real-Time Transport Protocol (RTP) and Real-TimeTransport Control Protocol (RTCP). The control flow sets up the VoIPcall signaling between the IP Phones or “endpoints” (residing atend-user location) and the call manager or proxy (residing at serviceprovider or HQ/central location). Once this handshake is established anddetermined to be valid and allowed by the security policy, then themedia (or voice data) can flow through. The term ‘flow’ as used hereinrefers to a VoIP flow, but it should be mentioned that the presentinvention is not limited to VoIP, nor is the invention limited to thetransfer of voice data. Rather, it will be apparent to those of skill inthe art that the concepts of the invention may be applied to and benefitany real-time communication protocol where groups of packets areforwarded between endpoints.

Voice packet delivery requires network bandwidth that is different fromdata delivery. During a call, voice delivery continuously consumes largeamounts of bandwidth at a constant bit rate. If the network does nothave available bandwidth for a voice call, the call's traffic can easilycongest the network. Congestion causes packets to be delayed or dropped,resulting in a poor quality of service.

According to one aspect of the invention, each network element (i.e.,router or other network device) performs traffic performancemeasurements on traffic per service class (e.g., DSCP) on its links.Traffic performance measurements may include a series of tests run onphysical and/or virtual links to measure traffic conditions of the path,the series of tests for measuring one or more of a performance, Qualityof Service (QoS), onset of congestion and connectivity state of thephysical or virtual links. The traffic measurements are used to whenselecting a path for assignment to a flow to route flows aroundcongested or otherwise undesirable paths. The present invention thusenables intelligent routing of flows in a network using real-timetraffic performance information.

Token bucket is an example of a traffic measurement method that may beused in the present invention. Token bucket is a control mechanism thatdictates when traffic can be transmitted, based on the presence oftokens in the bucket. The token bucket contains tokens, each of whichcan represent a unit of bytes. The network administrator specifies howmany tokens are needed to transmit a specified number of bytes; whentokens are present, a new flow is allowed to transmit traffic on thatpath. Thus, the number of tokens in a bucket is related to an ability ofa next hop device to service traffic.

According to one aspect of the invention, for each new flow that isreceived at a network element (NE), if there are no tokens in the bucketfor a primary next hop associated with the destination of the flow, theNE selects an alternate next hop from a list of alternate next hops andtransmits all packets belonging to that flow using the uncongested nexthop. Normally, the token bucket is configured so that it runs out oftokens before any significant amount of queuing of packets belonging toreal-time flows cures and thus is used to provide an early warning ofonset of congestion. A congestion mechanism that monitors token bucketpopulation may be used in the present invention for congestiondetection. Other traffic shaping methods such as the leaky bucketalgorithm and others known in the art may be readily substituted herein.

According to one aspect of the invention, Admission Control mechanismsmay be used to signal congestion to the endpoints in case all availablepaths are congested, improving the ability of the network to providehigh quality of service to traffic.

Potential paths to endpoints are identified at each NE prior to thereceipt of a flow by the NE. In one embodiment, the potential pathsinclude a primary path and one or more alternate paths, which are storedin a next hop list. The packet metering of real-time traffic isperformed all the time. When a new flow is received at a NE implementingthe present invention, traffic performance measurements are used todetermine a traffic state of the path and to select the appropriate nexthop NE from the list. For example, a path may be in a brownout state,where either one or both of the following are true: the measured trafficlevel on the physical or virtual link or service class is above anengineered limit, indicating onset of congestion; and the measuredperformance or QoS on the physical or virtual link or service class isbelow a measured level. When onset of congestion of the primary next hopis detected, an alternate uncongested next hop is selected. In oneembodiment, the alternate path is automatically assigned as the flowpath in the presence of congestion at the primary path. In a secondembodiment, traffic performance measurements are performed on thealternate next hop, and either the primary or alternate next hop isselected based on relative congestions of the paths.

Detected onset of congestion may indicate congestion of the serviceclass of physical links or Performance/QoS of the service class, forvirtual links. In other words, the present invention addresses theability to route application flows to alternate physical or virtuallinks based on onset of congestion and/or degradation of performance onthe selected link. This allows flows to achieve an acceptable level ofservice during brownout (path congestion and for virtual links if themeasured performance/QoS status is below the configured acceptableperformance level) or blackout (when the network or portion of thenetwork fails completely due to link or equipment failures or humanconfiguration errors) scenarios in the network, thereby increasing theavailability and resiliency of existing distributed IP networks.

Although standard routing protocols can detect and repair networkblackouts, those protocols are slow to converge once an outage isdetected. While such routing protocols will eventually route around ablackout, they will never route around a brownout; routing protocolsonly detect loss of link availability, not degraded performance. Thepresent invention overcomes the inadequacies of standard routingprotocols by identifying congested links (next hop) or performancedegradation, enabling flows to be routed around network brownout andblackout conditions quickly without dropping or mis-ordering packets.

FIGS. 1A and 1B are block diagrams of a network 10 in which the presentinvention may be implemented. A number of VoIP devices, such as VoIPphones 12, 14, 16, and 18, are coupled together via an IP network 10comprised of NEs 20, 22, 24, 26, and 28. A dashed line illustrates apath between VoIP phones 14 and 18, which traverses NEs 20, 22, 26, and28 on links 21, 23, and 25. The path is comprised of links 21, 23, and25 and is the primary path for transmissions between devices 20 and 28,wherein the primary path may have been selected in a variety of mannersknown to those of skill in the art. When endpoint 14 seeks to establisha connection with endpoint 18, after the session establishment phase,application data packets are forwarded between the endpoints. In FIG.1A, no onset of congestion is identified, and the flow is routed on theprimary path.

However, FIG. 1B illustrates the effect of onset of congestion beingidentified in the primary path at the time of the application's firstpacket transmission. On reception of the first application data packetfrom endpoint 12 heading to endpoint 16, NE 22 measured that its primarynext hop link 23 is congested or is experiencing performancedegradations, and therefore looks through its list of alternate anduncongested next hops available and it finds link 27. NE 22 enters flowstate and all application data packets from endpoint 12 heading toendpoint 16 will be routed on links 27 and 29.

FIG. 1C is a block diagram of a combined LAN/WAN configuration 50 thatshows two virtual links 53 and 55 between two Edge Routers 52 and 54.Based on the result of the Shortest Path Forwarding (SPF) algorithm, oneof the two virtual links 53 and 55 will be the primary link, e.g.,virtual link 53. Traffic performance measurement is continuallyperformed on both virtual links 53 and 55 to determine state. Based ontraffic performance measurement results, virtual link 53 is determinedto be in brownout state when a first packet arrives belonging to a newreal-time flow. Edge Router 52 selects an alternate virtual link 55 thatit uncongested and over which measured performance is good. All packetsbelonging to the flow associated with the first packet will follow thealternate virtual link 55 for the life of the flow or until that virtuallink fails.

Thus, the present invention provides a flow-based service that may beconfigured to run on physical or virtual links on a NE. Eachparticipating NE has the option to use an alternate outbound link,instead of the primary outbound link selected by the SPF algorithm,under blackout and brownout conditions. It may be used with or withoutAdmission Control mechanism, which can be enabled to limit the amount ofnew flows that can be allowed on all of the primary and alternateoutbound links and to protect them from going into a brownout state.

FIG. 2 illustrates several components that may be included in a NE 20capable of implementing the present invention. The components may beimplemented in hardware, software, or any combination thereof. Inaddition, it is understood that the components are representative ofcertain functions that may be performed by the NE 20; similar functionsmay be represented in embodiments that are differently delineated.Accordingly, the present invention is not limited by the embodimentillustrated in FIG. 2.

NE 20 includes packet classification logic 30, flow cache 32, real-timetraffic performance measurement 34 (includes packet metering,connectivity detection and QoS/performance measurement), and firstpacket processing logic 36. In one embodiment, the packet classificationlogic 30 generates a key using the endpoint information and serviceclass information. For example, the endpoint information may include oneor more of a Source IP address (SRC IP), Destination IP address (DSTIP), SRC Port, DST Port, and service class information may includeDifferentiated Services Code Point (DSCP). The present invention is notlimited to any particular endpoint information or service classinformation when generating a key. For example, port numbers may be usedfor voice gateways that support many flows to/from the same IP address.

The key is used to index flow cache 32, which stores flow stateincluding an outbound link for each known or previously processed flow.The existence of flow state in the flow cache indicates whether the flowhas already been processed by first packet processing logic 36. If flowstate is found, the outbound link included in the flow state is used totransmit the packet.

If flow state is not found (new flow or network failure condition), thena MISS signal is forwarded to the first packet processing logic 36 toestablish the flow state by identifying an outbound link for the flow.During the selection of the outbound link, results of trafficperformance measurements of congestion, connectivity, and performance(provided by real-time traffic performance measurement 34) are used inselection of a valid outbound link. A valid outbound link is selectedfrom one of a primary link and alternate links and is one that meets theabove measurement criteria and lowest cost selection. This outbound linkis cached in the flow state for subsequent packets of this flow and thepacket is then sent out on this outbound link. All following packetsbelonging to that flow will be forwarded on that outbound link.

FIG. 3 is a flow diagram illustrating several steps of a process 100that may be performed by a NE, e.g., NE 20, supporting the presentinvention. As described above with regard to FIG. 2, as packets arereceived at the NE, they are classified to generate a key at step 104,and at step 108, the flow cache is indexed to determine whether the flowhas been processed. If the flow has been processed, then at step 110,the outbound link is retrieved from the flow state. At step 112, trafficperformance measurement is performed on the outbound link, and a pathstate of the link is identified, wherein the states may include Up,Brownout, Blackout, etc. Thus, a current picture of the trafficcongestion on the path is maintained. More detail on congestionmeasurement is provided below. Once the measurement is complete, thepacket is transmitted on the identified outbound link at step 125, andthe process returns to step 102, awaiting new packets.

If at step 108 there was no flow state in the flow cache, the processproceeds to step 122, where an outbound link is identified from a listof primary and alternate next hops. The primary next hop may be aneighboring NE identified to be a most desirable neighboring NE based onnetwork information and an executing protocol. In an exemplaryembodiment, each of the NEs are IP NEs, which run Interior GatewayProtocols (IGPs) such as OSPF (Open Shortest Path First) or IS-IS(Intermediate System-Intermediate System). OSPF and IS-IS are also knownas link state routing protocols. In link state routing protocols, eachNE composes a link state information packet or advertisement thatdescribes its links to neighbors and the state of each link (up ordown). This information is distributed to every NE in the domain usingan efficient flooding algorithm that minimizes traffic overhead andensures that each NE receives every link state advertisement. When theflooding is complete, every NE in the routing domain will haveaccumulated a database consisting of all the advertised links and theirstates generated by each participating NE in the topology. The linkstates may be used to identify the ‘best’ or primary next hop device ina path to a destination.

The outbound link is identified at step 122 based on traffic performancemeasurements of primary and alternate links, after which further packetmetering and performance measurement is performed in step 112. Trafficperformance measurement may take many forms, known to those of skill inthe art, and the present invention is not limited to a particular methodof traffic performance measurement, nor is it limited to any particulartraffic thresholds. Traffic performance measurements may reflect morethan simple congestion; for example, path states may be reserved fordifferent combinations of congestion, QoS, connectivity, and other linkand/or path related states. For example, in one embodiment, UP state mayindicate no-congestion, QoS performance good, and connectivity OK.Brownout state may indicate congested or QoS/performance below definedlimit and connectivity OK. Blackout state may indicate no connectivity(down) or very bad QoS/performance.

Depending on the current Traffic Performance Measurement results for theprimary and alternate links at step 112, a decision is made at the timeof first packet processing about whether to route the new flow ontospecific alternate links or to allow it to traverse a primary link.

The list of alternate paths to any given destination can be identifiedin a variety of ways. One mechanism for selecting alternate paths to adestination is through the application of Reliable Alternate Paths forIP Destination (RAPID) identification processes, as described in NetworkWorking Group Internet Draft “Basic Specification for IP Fast Reroute:Loop-free Alternates” by A. Atlas, draft-ietf-rtgwg-ipfrr-spec-base-05”,February 2006, incorporated herein by reference, or as described in theNetwork Working Group Internet Draft “IP Fast Reroute Framework”,draft-ietf-rtgwg-ipfrr-framework-05.txt, March 2006 by Shand, alsoincorporated herein by reference.

The typical use of (one or more) RAPID-based alternate next hops is toredirect traffic on a hop-by-hop basis, in case of primary next hop orlink failure, thus reducing network down-time. In contrast, the presentinvention uses alternate path information provided by RAPID forredirecting traffic at a flow level granularity. Flows may be forwardedto the alternate path in parallel with the primary path to provide loadsharing capability for VoIP traffic.

While RAPID is a preferred method of identifying alternate paths, it isnot a requirement of the present invention that RAPID be used; ratherpre-configured alternate link(s) corresponding to primary link may besubstituted herein without affecting the scope of the invention.Preconfigured links may be used in the case of tunnels where dynamicrouting is not being used for tunnel routes.

There are a variety of factors that may be considered when selecting oneof a primary or alternate path for a flow, and the present invention isnot limited to the consideration of any particular factors. By way ofexample, a process for selecting a link from the set of primary andalternate links may examine the cost as well as the results of thetraffic performance measurement of the link. The lowest cost link not inbrownout state is first selected. When there are more than one lowestcost links, not in brownout state, then load sharing is performed amongthem. An Equal Cost Multi Path (ECMP)-like mechanism may be used todistribute flows among these equal cost links, proportional to thecapacity of each link for that service class. When all links are inbrownout state, packets for new flows must still be forwarded at therisk of causing performance degradation. Packets for such flows areforwarded on the lowest cost link. If there is more than one lowest costlink, load-sharing is performed among them to distribute new flows.

In the presence of a blackout condition, the failed link is removed fromthe set of primary and alternate links for that destination. Flows thatwere associated with the failed link are now moved to existing links.When, as a result of dynamic routing, a new link is added to the set ofprimary and alternate links for a destination, new flows may be assignedto the new link using the above described mechanisms.

Thus, the identified outbound link is saved in the newly created flowstate in step 122 for future packets for this flow. Traffic performancemeasurement is performed on this link and the packet is transmitted outon this link. The process returns to step 102 where more packets areprocessed as they are received.

Accordingly, a method and apparatus have been shown and described thatprovide the ability to route new flows onto one or more alternate linksif the primary link goes into a blackout or a brownout state. Thepresent invention may be configured on physical or virtual links on a NEto enhance the forwarding of packets using the primary link and one ormore alternate links to a destination. With such an arrangement, a NEcan mitigate traffic congestion and bandwidth unavailability on a perservice class basis by providing alternate physical or virtual links,and automatically route new flows resulting from blackouts, brownouts,or normal traffic with flow quality of service guarantee. The NE canensure delivery of traffic using multiple alternate outbound links inthe presence of a brownout condition on the primary outbound linkwithout reordering of packets within flows, except for flows resultingfrom blackouts.

Having described various embodiments of the invention, it will beappreciated that many of the above figures are flowchart illustrationsof methods, apparatuses (systems), and computer program productsaccording to an embodiment of the invention. It will be understood thateach block of the flowchart illustrations, and combinations of blocks inthe flowchart illustrations, can be implemented by computer programinstructions. These computer program instructions may be loaded onto acomputer or other programmable data processing apparatus to produce amachine, such that the instructions which execute on the computer orother programmable data processing apparatus create means forimplementing the functions specified in the flowchart block or blocks.These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the function specified in the flowchart block or blocks.The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing the functions specified in theflowchart block or blocks.

Those skilled in the art should readily appreciate that programsdefining the functions of the present invention can be delivered to acomputer in many forms; including, but not limited to: (a) informationpermanently stored on non-writable storage media (e.g., read only memorydevices within a computer such as ROM or CD-ROM disks readable by acomputer I/O attachment); (b) information alterably stored on writablestorage media (e.g., floppy disks and hard drives); or (c) informationconveyed to a computer through communication media, for example, usingbaseband signaling or broadband signaling techniques, including carrierwave signaling techniques, such as over computer or telephone networksvia a modem.

The above description and figures have included various process stepsand components that are illustrative of operations that are performed bythe present invention. However, although certain components and stepshave been described, it is understood that the descriptions arerepresentative only; other functional delineations or additional stepsand components can be added by one of skill in the art, and thus thepresent invention should not be limited to the specific embodimentsdisclosed. In addition, it is understood that the variousrepresentational elements may be implemented in hardware, softwarerunning on a computer, or a combination thereof.

While the invention is described through the above exemplaryembodiments, it will be understood by those of ordinary skill in the artthat modification to and variation of the illustrated embodiments may bemade without departing from the inventive concepts herein disclosed.Accordingly, the invention should not be viewed as limited except by thescope and spirit of the appended claims.

What is claimed is:
 1. A method for selecting a path for a flowcomprising a plurality of packets, said method comprising: performing,by a network element, traffic performance measurements on a primary pathto determine congestion, Quality of Service (QoS), and connectivity; andselectively routing, by the network element, the flow to an outboundlink around the primary path in response to a comparison of thecongestion, QoS, and connectivity of the primary path with congestion,QoS, and connectivity of an alternate path.
 2. The method of claim 1,wherein the traffic performance measurements indicate that the primarypath is in a blackout condition indicating a failure in the primarypath.
 3. The method of claim 2, wherein the failure is one of a linkfailure or a node failure.
 4. The method of claim 1, wherein the flowtraverses one of a local area network or a wide area network.
 5. Themethod of claim 1, further comprising: identifying, by the networkelement, traffic performance measurements on all paths to a destinationof the flow; and selectively routing, by the network element, a new flowonto the primary path based on the traffic performance measurements ofall paths.
 6. The method of claim 1, further comprising: storing, by thenetwork element, for each known endpoint, a primary next hop nodeidentifier and one or more alternate next hop node identifiers.
 7. Themethod of claim 6, wherein the outbound link is identified using a listof primary and alternate next hop identifiers.
 8. The method of claim 1,further comprising: receiving, by the network element, a packet;generating, by the network element, a key using information in thepacket; determining, by the network element, using the key, whether theoutbound link for forwarding the packet has been previously identified;and in response to a determination that the outbound link has not beenpreviously identified, identifying and selecting the outbound link. 9.The method of claim 8, wherein determining whether the outbound link hasbeen previously identified comprises indexing a flow cache using thekey, wherein a flow state stores the outbound link.
 10. The method ofclaim 9, wherein generating the key comprises use of at least one of aSource IP address (SRC IP), a Destination IP address (DST IP), aDifferentiated Service Code Point (DSCP), a Source Port, or aDestination Port included in the packet.
 11. A network elementcomprising: real-time traffic performance measurement logic forperforming traffic performance measurements on a primary path todetermine congestion, Quality of Service (QoS), and connectivity; andfirst packet processing logic for selectively routing a flow to anoutbound link around the primary path in response to a comparison of thecongestion, QoS, and connectivity of the primary path with congestion,QoS, and connectivity of an alternate path.
 12. The network element ofclaim 11, wherein the traffic performance measurements indicate that theprimary path is in a blackout condition indicating a failure in theprimary path.
 13. The network element of claim 12, wherein the failureis one of a link failure or a node failure.
 14. The network element ofclaim 11, wherein the flow traverses one of a local area network or awide area network.
 15. The network element of claim 11, wherein thereal-time traffic performance measurement logic: identifies trafficperformance measurements on all paths to a destination of the flow; andselectively routes a new flow onto the primary path based on the trafficperformance measurements of all paths.
 16. The network element of claim11, further comprising: a memory for storing, for each known endpoint, aprimary next hop node identifier and one or more alternate next hop nodeidentifiers.
 17. The network element of claim 16, wherein the outboundlink is identified using a list of primary and alternate next hopidentifiers.
 18. The network element of claim 11, further comprising:packet classification logic for receiving a packet, generating a keyusing information in the packet, and determining, using the key, whetherthe outbound link for forwarding the packet has been previouslyidentified, wherein the first packet processing logic identifies andselects the outbound link in response to a determination that theoutbound link has not been previously identified.
 19. The networkelement of claim 18, wherein determining whether the outbound link hasbeen previously identified comprises indexing a flow cache using thekey, wherein a flow state stores the outbound link.
 20. The networkelement of claim 19, wherein generating the key comprises use of atleast one of a Source IP address (SRC IP), a Destination IP address (DSTIP), a Differentiated Service Code Point (DSCP), a Source Port, or aDestination Port included in the packet.